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TMAGE AND VIDEO OUAT JTY MEASUREMENT 

FIELD OF THE INVENTION 

The present invention relates to the measurement of image and video quality. 
The mvention is particularly useful for, but not necessarily Umited to aspects of the 
measurement of image and video quality without reference to a reference image Cno- 
reference" quality measurement). 

BACKGROUND ART 

Images, whethea: as individual images, such as photographs, or as a series of 
images, such as ftames of video are increasingly transmitted and stored electronically, 
whether on home or lap-top computers, hand-held devices such as cameras, mobile 
telephones, and personal digital assistants (PDAs), or elsewhere. 

Although memories are getting larger, there is a continuous quest for reducing 
images to as little data as possible to reduce transmission time, bandwidth requirements 
or memory usage. This leads to ever improved intra- and inter-unage compression 
techniques. 

Inevitably, most such techniques lead to a loss of data in the de-compressed 
images. The loss from one compression technique may be acceptable to the human eye 
or an electronic eye, whilst from another, it may not be. It also varies according to the 
sampling and quantization amounts chosMi in any technique. 

To test compression techniques, it is necessary to determine the quality of 1h.e end 
result. That may be achieved by a human judgement, although, as witihi all things, a more 
objective, empirical approach may be preferred. However, as the ultimate target for an 
image is most usually the human eye (and brain), the criteria for deteraiining quality are 
generally selected accorduig to how much the particular properties or features of a 
decompressed image or video are noticed. 
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For instance, distortion caused by compression can be classified as blockiness, 
blurring, jaggedness, ghost figures, and quantization errors. Blockiness is one of the 
most annoying types of distortion. Blockiness, also knovra as the blocking effect, is one 
of the major disadvantages of block-based coding techniques, such as JPEG or MPEG. 
5 It results from intensity discontinuities at the boundaries of adjacent blocks in the 
decoded image. Blockiness tends to be a result of coarse quantization in DCT-based 
unage compression. On the other hand, die loss or coarse quantization of high frequency 
components in sub-band-based image compression (sudi as JPEG-2000 image 
compression) results in pre-dominant blurring effects, 

10 

Various attempts to measure image quality have been proposed. However, in 
most cases it is with reference to a non-distorted reference image because it is easier to 
explain quality deterioration with reference to a reference image. Even then, it has been 
found that it is very difficult to teach a machine to emulate the human vision system, 
15 even with a reference image, and it is even more difficult when no reference is available. 
On the oflier hand, human observera can easily assess the quality of ima^ witiiout 
requiring any reference undistorted image/video. 

Wang, Z., Sheikh, H.R., and Bovik, A.C., "No-reference perceptual quaUly 
20 assesatnent of JPEG compressed images", International Conference on Image 

Processing, September 2002, proposes a no-reference perceptual quality assessment 
metric designed for assessing JPEG-compressed im^es- A blockiness measure and two 
bluning measures are combuxed into a single model and the model parameters are 
estunated by fitting the model to the subjective test data. However, this method does not 
25 seem to perform well on images where biockmess is not the predominant distortion. 

Wu, H.R. and Yuen, M., "A generalize block-edge impairment metric for video 
coding," IEEE Signal Processing Letters., Vol. 4(1 1), pp.3 17-320, 1997, proposes a 
block-edge impairment metric to measure blocking in images and -sddeo without 
30 requiring the original image and video as a comparative reference. In this method, a 
weighted sum of squared pixel gray level differences at 8x8 block boundaries is 
computed. The weighting function fer each block-edge pixel difference is designed 
using local mean and standard deviations of the gray levels of the pixels to the left and 
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right of the block boundary. Again, this method does not seem to perform well on 
images where blockiness is not the predominant distortion. 

Meesters, L., and Martens, J.B., "A single-ended blockiness measure for JPEG- 
5 coded images". Signal Processmg, Vol. 82, 2002, pp. 369-387, proposes a no-reference 
(single-ended) blockiness measure for measuring the image quality of sequential 
baseline-coded JPEG images. Hiis method detects and analyses edges based on a 
Gaussian blimced edge model and uses two separate one-dimensional Hermite transforms 
along the rows and columns of the image. Then, the unknown edge parameters are 
10 estimated from the Hermite coefficients. This method does not seem to perform well on 
images where blocldness is not tiie predominant distortion. 

Lubin, J., BriU, MM., and Rca, A.P., "Method and apparatus for estimating video 
quality without usmg a reference video", US Patent 6285797, Sep. 2001. proposes a 

1 5 method for estimating digital video quality without using a reference video. This method 
requires computation of optical flow and specific techniques which include: (1) 
Extraction of low-amplitude peaks of the Hadamard transform, at code-block 
periodicities (useful m deciding if there is a broad uniform area with added JPEG-like 
blockiness); (2) Scintillation detection, usefiil for determining likely artefects in the 

20 neighbourhood of moving edges; (3) Pyramid and Fourier decomposition of the agnal to 
reveal macroblock artefacts (MPEG-2) and wavelet ringing (MPEG-4). This method is 
very computationally intensive and time consuming. 

Bovik, A.C., and Liu, S., «DCT-domain blind measurement of blockmg artifacts 
25 in DCT-coded images", IEEE International Conference on Acoustic, Speech, and Signal 
Processing, Vol. 3, May 2001, pp. 1725-1728, proposes a method for blind (i.e. no- 
reference) measurement of blocking artefects in the DCT-domain. In this approach, a 
8x8 block is constituted across any two adjacent 8x8 DCT blocks and the blocking 
artefact is modelled as a 2-D step function. The amplitude of the 2-D step function is 
30 then extracted from tiie newly constituted block. This value is then scaled by a function 
of the background activity value and the average value of the block and the final value of 
all the blocks are combmed to give an overall blocking measure. Again, this method 
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does not seem to perform well on images where blockiness is not the predominant 
distortion. 

Wang, Z., Bovik, A.C., and Evans, B.L., "Blind measurement of blocking 
artifacts in images", IEEE International Conference on Image Processing, Sep. 2000, pp. 
981-984, proposes a method for measuring blocking artefacts in an image without 
requiring an original reference image. The task here is to detect and evaluate the power 
of the image. A smoothly varying curve is used to approximate the resulting power 
spectrum and the powers of the frequency components above this curve are calculated 
and used to determine a final blockiness measure. Again, this method does not seem to 
perform well on images v^^iere blockiness is not the predominant distortion. 

SUMMARY OF THE INVENTION 

According to one aspect of the preseaat mvention, there is provided ^paratus for 
determining a measure of image quality of an image. The apparatus includes means for 
determining a blockmess mvisibility measure of the image; means for determining a 
colour richness measure of the image; means for detemuning a sharpness measure of the 
image; and means for providing the measure of hnage quality of the image based on the 
blockiness invisibility measure, the colour richness measure and the slwipness measure 
of the image. 

According to a second aspect of the present invention, there is provided apparatus 
for determining a blodkiness mvisibility measure of an image. The apparatus comprises: 
means for averaging differences in colour values at block boundaries within the image; 
means for averaging differences in colour values between adjacent pixels; and means for 
providing the blockmess Divisibility measure based on averaged differences in colour 
values between adjacent pixels and averaged differences in colour values at block 
boundaries vdthin the image. 

According to a third aspect of the present invaition, there is provided apparatus 
for determining a colour richness measure of an image. The apparatus comprises: means 
for determining the probabilities of individual colour values within the image; means for 
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detenranmg the products of the probabilities of individual colour values and the 
logarithms of the probabilities of individual colour values; and means for providmg the 
colour richness measure based on ffae sum of the products of the probabilities of 
individual colour values and the lo^thms of the probabilities of individual colour 
5 values. 

According to a fourth aspect of the present uivention, there is provided apparatus 
for determining a sharpness measure of an rniage. The ^paratus comprises: means for 
determining differences m colour values between adjacent pixels within the image; 

10 means for deteraiining the probabilities of individual colour value differences within the 
image; means for determining the products of the probabiUties of individual colour value 
differences and the logarithms of the probabiUties of individual colour value differences; 
and means for providing the sharpness measure based on the sum of the products of the 
probabilities of individual colour value differences and flie logarithms of the probabiUties 

15 of individual colour value differences. 

According to a fifth aspect of the present invention, there is provided apparatus 
for determining a measure of image quaUty of an image within a sequence of two or 
more images. Hie S5)paratus comprises: apparatus according to the first aspect; and 
20 means for determining a motion activity measure of the image within tiie sequence of 
images. 

According to a axth aspect of the present iavention, thrare is provided apparatus 
for detennining a motion activity measure of an unage within a sequence of two or more 

25 unages. The apparatus comprises: means for determining differences in colour values 
between pixels within the image and corresponding pixels in a preceding image within 
the sequence of images; means for determining the probabiUtiKi of individual colour 
value differences between the image and the preceding unage; means for determming the 
products of the probabiUties of individual colour value differences and the logarithms of 

30 the probabilities of individual colour value differences; and means for providing the 
motion activity measure based on the sum of the products of the probabiUties of 
individual colour value differences and the logarithms of fee probabUities of individual 
colour value differences. 
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According to a seventh aspect of the present invention, there is provided 
apparatus for determining a measure of video quality of a sequence of two or more 
hnages* The apparatus comprises: apparatus according to the first or fifth aspects; and 
5 means for providing the measure of video quality based on an average of the image 
quality for a plurality of images within the sequence of two or more images. 

According to an eighth aspect of the present invention, there is provided a 
method of determining a measure of image quality of an image. The method comprises: 
1 0 determining a blockiness invisibility measure of the image; determinmg a colour richness 
measure of the image; determining a sharpness measure of the image; and providing the 
measure of image quality of the hnage based on the blockiness invisibility measure, the 
colour richness measure and the sharpness measure of the image. 

1 5 According to further aspects of the present invention, there are provided methods 

corresponding to the second to seventh aspects. 

According to yet further aspects of the present invention, there are provided 
computer program products operable according to the eighth aspect or the further 
20 methods and computer program products which when loaded provide apparatus 
according to the first to seventh aspects. 

At least one aspect of the invention is able to provide an image quality 
measurement system which determines various features of an image that relate to the 
25 quality of the image in terms of its appearance. The features may include one or more 
of: Hie image's blockiness invisibility, the image's colour richness and the image's 
sharpness. These may all be obtained without use of a reference image. The one or 
more deteraiined features, with or without other features, are combined to provide an 
image quality measure. 

30 

INTRODUCTION TO THE DRAWINGS 
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The present invention may be further understood from the following description 
of non-limitative examples, with reference to the accompanying drawings, in which:- 

Figure 1 is a block diagram of an image quality measurement system, according 
5 to a first embodiment of the invention; 

Figure 2 is a flowchart relating to an exemplary process in the operation of the 
systemof Figure 1; 

10 Figure 3 is a flowchart relatmg to an exemplary process in the operation of one of 

the features of Figure 1, which appears as a step of Figure 2; 

Figure 4 is a flowchart relating to an exemplary process in the operation of 
another of the features of Figure 1, which appears as a step of Figure 2; 

15 

Figure 5 is a flowchart relating to an exemplary process in the opemtion of again 
another of the features of Figure 1, which appears as a step of Figure 2; 

Figure 6 is a block diagram of a video quality measurement system, according to 
20 a second embodiment of the invention; 

Figure 7 is a flowchart relating to an exemplary process in the operation of the 
system of Figure 1; and 

25 Figure 8 is a flowchart relating to an exemplary process in the operation of one of 

the features of Figure 6, which appears as a step of Figure 7. 

DESCRIPTION 

30 Where the same reference numbers appear in more than one Figure, they are 

being used to refer to the same components and should be imderstood accordingly. 
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Figure 1 is a block diagram of an image quality measurement system 1 0, 
according to a first embodiment of the invention. An exemplary process in the operation 
of the system of Figure 1 is described with reference to Figure 2. 

5 An image signal I, corresponding to an image whose quality is to be measured, is 

input (step SI 10) to an image quality measurement system 10. The image signal I is 
passed, in parallel, to three modules, an image blockiness invisibiUty feature extraction 
module 12, an image colour richness feature extraction module 14 and an image 
sharpness feature extraction module 16. 

10 

Each of these three above-mentioned modules 12, 14, 16 performs a different 
function on the image signal I to produce its own output signal. The image blockiness 
invisibility feature extraction module 12 determines a measure of tibie image blockiness 
invisibiUty from the image signal I and ou^uts a blockiness invisibiUty measure B (step 
1 5 S 120). The image colour richness feature extraction module 14 determines a measure of 
the image colour richness from the image signal I and outputs an image colour richness 
measure R (step S130). The unage sharpness feature extraction module 16 determines a 
measure of the image sharpness from the image signal I and oulputs an image sharpness 
measijre S (step S140). 

20 

The three output signals B, R, S are input together mto an unage quality model 
module 18, where they are combuied to determine an image quaUty measure Q (step 
S160), which is output (step S170). 

25 1 (i) Image Blockiness hivisibilitv Feature Extraction 

The image blockiness invisibility feature measures the invisibiUty of blockiness 
in an image without requiring a reference undistorted original image for comparison. It 
contrasts wth image blockiness, which measures the visibiUty of blockiness. Thus, by 
30 definition, an image blockiness invisibility measure gives lower values when image 
blockiness is more severe and more distinctly visible and higher values when image 
blockiness is very low or does not exist in an image. 
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The image blockiness invisibility measure, B, is made up of two components, a 
numerator D and a denonainator C, which in turn are made up of 2 separate components 
measured in both the horizontal x-direclion and the vertical y-direction. The horizontal 
and vertical components of D, labelled Dh and D,, and the horizontal and vertical 
5 components of C, labelled Ch and Cy, ate defined as follows: 

■"*^;j; = lx = l 

where 

1 0 iix,y) daiotes the colour value of the input image I at pixel location (x,y), 

H is the height of the image, 
W is the width of the image, 

xe[l,fr],and 

y^ixm. 



Similarly, 

\ H W 



where 



20 dy{x,y) = lix.y + 1) - I{x,y) . 



The horizontal and vertical components of D are computed from block 
boundaries interspaced 8 pixels apart in the horizontal and vertical directions, 
respectively. 



25 
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The blockiness invisibility measiire B, composed of 2 separate components Bh 
and Bv, is defined as follows: 

_ g(Ch) 



A parameterisation of fhe form: 



enables B to correlate closely with human visual subjective ratings. The 
10 parameters are obtained by correlating with human visual subjective ratings via an 

optunisation process such as Hooke and Jeeve's pattern-search method (Hooke R., Jeeve 
T. A., "Direct Search" solution of numerical and statistical problems. Journal of the 
associate computing machinery^ Vol, 8, 1961, pp- 212-229). 



15 An exemplary process in the operation of the image blockiness invisibility feature 

extraction modxile 12 of Figure 1, which appears as step S120 of Figure 2, is described 
with reference to Figure 3. In this process, for the input image, differences are 
determined between the colour values of adjacent pixels at block boundaries, in a furst 
direction (step S121). An average difference for every block in the furst direction for 

20 every layer of pixels in the second direction is determined (step S 1 22). Additionally the 
average difference between the colour values of adjacent pixels in the first direction for 
every pixel is determined (step SI 23). Functions are applied to these two averages for 
the first direction, fi-om steps S122 and S123, to provide a blockiness invisibility 
component for the first direction (step S124). For instance the average from step S123 is 

25 raised to the power of a first constant, while the average from step 122 is raised to the 
power of a second constant, and the component is determined as a ratio of the two raised 
averages. 
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Dififerences are also determined between the colour values of adjacent pixels at 
block boundaries, in the second direction (step S125). An average difference for every 
block in the second direction for every column of pixels in the first direction, is also 

5 determined (step S126). Additionally the average difference between the colour values 
of adjacent pixels in the first direction for every pixel is determined (step S 127). 
Functions are applied to these two averages for the second direction, from steps S126 
and S127, to provide a biockiness invisibility component for the second direction (step 
S128). For instance the average from step S127 is raised to the power of the first 

10 constant, while the average firom step 126 is raised to the power of the second constant, 
and the component is determined as a ratio of the two raised averages* 

The biockiness invisibility components for the two directions, firom steps S124 
and S128, are averaged and the average is output (step S129) as the biockiness 
1 5 invisibility measure B. 

1 Tmape Colour Richness Feature Extraction 

The image colour richness feature measures the richness of an image's content. 
20 This colour richness measure gives higher values for images which are richer in content 
(because it is more richly textured or more colourful) compared to images which are very 
dull and unlively* Hiis feature closely correlates with the human perceptual response 
which tends to assign better subjective ratings to more lively and more colourfiil images 
and lower subjective ratings to dull and unlively images. 

25 

The image colour richness measure can be defined as: 

R = - Zp(Ologe(p(0) 

where 

Vi 
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i is a particular colour (eithea: the luminance or the chrominance) value, 
[0,255], 

N(i) is the nxunber of occmrence of i in the image, and 

p(i) is the probability or relative frequency of i appearing in the image. 

This image colour richness measure is a global image-quality feature, computed 
from an ensemble of colour values' data, based on the sum, for all colour values, of the 
product of the probability of a particular colour and the logarithm of the probability of 
the particular colour. 

An exemplary process in the operation of the image colour richness feature 
extraction module 14 of Figure 1, which appears as step S130 of Figure 2, is described 
with reference to Figure 4. In this process, for the input unage, the probability or relative 
frequency of a colour is determined for each colour within the image (step SI 32)- For 
each colour a product of the probability of that colour and the natural logarithm of the 
probability of that colour, is determined (step S134). These products are summed for all 
colours (step S136), with the negative of that sum is output (step S138) as the image 
colour richness measiare R. 

1 (ill) Image Sharpness Extraction Feature 

The image sharpness feature measures the sharpness of an image's content and 
assigns lower values to blurred images (due to smoothing or motion-blurring) and higher 
values to sharp images. 

The image sharpness measure has 2 components, Sh and Sv, measured in both the 
horizontal x-dhrection and the vertical y-direction. 

The component of the image sharpness measure in the horizontal x-direction, Sh, 
is defined as: 



where 




dk y) = + 1, j^) - 7) , 



/(x, 3;) denotes the colour value of the input image I at pixel location (x,y), 
H is the height of the image, 
W is the width of the image, 



dh is the difference values in the horizontal x-direction, 

N(dh) is the mnnber of occurrences of dh among all the difference values in the 
horizontal x-direction, and 

p(dh) is the probability or relative frequency of dh appearing in the difference 
values in the horizontal x-direction» 

Similarly, the second component of the image sharpness measure in the vertical 
y-direction, Sv? is defined as: 



dv{^,y) = li^yy + 1) - K^^y) 

dv is the difference values in the vertical y-direction, 

N(dv) is the number of occurrences of dv among all the difference values in the 
horizontal y-direction, and 

p(dv) is the probability or relative firequency of dy appearing in the difference 
values in the horizontal y-direction. 



X G [UW] , 



where 
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The image sharpness measxire is obtained by combining the horizontal and 
vertical components, Sh and Sv, using the following relationship: 

This image sharpness measure is a global image-quality feature, computed from 
an ensemble of differences of neighbouring image data, based on the sum, for all 
differences, of the product of the probability of a particular difference value and the 
logarithm of the probability of the particular difference value, 

An exemplary process in the operation of the image sharpness feature extraction 
module 16 of Figure 1, which appears as step S140 of Figure 2, is described with 
reference to Figure 5. In this process, for the input image, differences are determined 
between the colour values of adjacent pixels in a first direction (step S141). The 
probability or relative frequency of each colour value difference in the first direction is 
determined (step S142). For each colour value difference in the first direction a product 
of the probability of that difference and the natural logarithm of the probability of that 
difference, is determined (step S143). These products are summed for all colour value 
differences in the first direction (step S144). Differences are also determined between 
the colour values of adjacent pixels in a second direction (step S 145). The probability or 
relative frequency of each colour value difference in the second direction is determined 
(step S146). For each colour value difference in the second direction a product of the 
probability of that difference and the natural logarithm of the probability of that 
difference, is determined (step S147). These products are summed for all colour value 
differences in the second direction (step 8148). The negatives of the two sums, from 
steps S144 and S148, are averaged (step S149) and the average is output (step S150) as 
the image sharpness measure S, 

1 f iv^ Im a ge Quality Measurement 



The image-quality measures B, R, S are combined into a single model to provide 
an image quality measure. 
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An image quality model which has been found to give good results for greyscale 
images is expressed as: 



or as 



^1 



Q = a + /3 




(1) 
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The parameters, a, p, yi (for i = 1 4), and 8 are obtained by. an optimisation 

process, such as Hooke and Jeeve*s pattern-search method, mentioned earlier, based on 
the comparison of the values generated by the model and the perceptual image quality 
ratings obtained in image subjective rating tests so that the model emulates the function 
of human visual subjective assessment capability. 



Thus the quality measure is a sum of three components. The first component is a 
fu-st constant. The second component is a product of the sharpness measure, S, raised to 
15 a jBrst power, the image blockiness invisibility measure, B, and a second constant. The 
third component is a product of the richness measure, R, raised to a second power, and a 
third constant. 

For colour images, the same algorithm (1) described above is applied to each of 
20 the three colour components, luminance Y, and chrominance Cb and Cr, separately, and 
the results are combined as follows to give a combined final image quality score: 



These parameters, a, p and 8 can similarly be obtained by an optimisation 
25 process, based on the comparison of the values generated by the colour model and the 
perceptual image quality ratings obtained in image subjective rating tests, so that the 
model emulates the function of human visual subjective assessment capability. 
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The above image quality model is just one example of a model to combine the 
image-quality measures to give an image quality measure. Other models are possible 
instead. 

Figure 6 is a block diagram of a video quality measurement system 20, according 
to a second embodiment of the invention. 

A video signal V, corresponding to a series of video images (frames) whose 
quality is to be measured, is input to a video quality measurement system 20. The 
current image of the video signal V passes, in parallel, to a delay unit 22 and to four 
modules: an hnage blockiness invisibility feature extraction module 12, an image colour 
richness feature extraction module 14, an image sharpness feature extraction modide 16 
and a motion-activity feature extraction module 24. 

The delay xmit 22 has a delay timing equivalent to one frame, then outputs the 
delayed image to the motion-activity feature extraction module 24, so lhat it arrives in 
parallel with the next image. 

The image blockiness invisibility feature extraction module 12, the image coloin: 
richness feature extraction module 14 and the image sharpness feature extraction module 
16 operate on the input video frame in the same way as on the input image in the 
embodiment of Figure 1, to produce similar output signals B, R, S. 

The motion-activity feature extraction module 24 determines a measure of the 
motion-activity feature from the current image of the video signal V and outputs a 
motion-activily measiure M. 

The four output signals B, R, S, M are input together into a video quality model 
modvde 26, where they are combined to produce a video quality measure Qy. 

An exemplary process in the operation of the system of Figure 6 is described with 
reference to Figure 7. The series of images is input into the system 20, one after the 
other (step S210). A frame count "N" is initiated at '*N = 0" (step S212). The frame 
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count is then increased by one (i,e. "N = N + 1"), in the iBrst pass-through of this step 
that means this is frame number 1 of the video segment whose quality is being measured. 

For the current frame, the process produces the image blockiness invisibility 
5 measure B, the image colour richness measure R and the image sharpness measure S 
(steps S120, S130, S140) m the same way as described with reference to Figures 1 to 5. 
For the current frame, the process also determines a motion-activity measure M, based 
on the current frame and a preceding frame (in this embodiment it is the immediately 
preceding frame) (step S260). Image quality for the cxjrrent frame is then determined in 
10 the video quality model module 26 (step S270), based on the image blockiness 
invisibility measure B, the image colour richness measure R, the image sharpness 
measure S and the motion-activily measure M for the current frame, 

A detemiination is made as to whether the incoming video clip, or the portion of 
1 5 video whose quality is to be measured has finished (step S272), If it has not finished, Ihe 
process returns to step S214 and the next frame becomes the current frame. If it is 
determined at step S272 that there are no more frames to process, the image quality 
results from the individual frames are used to deteraiine the video quality measure (step 
S280) for the video sequence, which video quality measure is then output (step S290). 

20 

2m Motion-Activity Feature Extraction 

The motion-activity feature measures the contribution of the motion in the video 
to the perceived image quality, 

25 

The motion-activity measure, M, is defined as follows: 
p(df)lO 

where 
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mdf) 

^ f y) = ^(^' 0 - ^(^3 y.t-i) 

I(x,y,t) is the colour value of the image I at pixel location I(x,y) and at frame t, 
I(x,y,t-1) is the colour value of the image I at pixel location (x,y) and at frame t-1, 
5 df is ttie ftame difference value, 

N(df) is the number of occurrence of df in the image-pair, and 

p(df) is the probability or relative frequency of df appearing in the image-pair. 

This motion-activity measure is a global video-qualily feature computed from an 
10 ensemble of colour differences between a pair of consecutive frames, based on the sum, 
for all differences, of the product of the probabilily of a particular difference and the 
logarithm of the probability of the particular difference. 

An exemplary process in the operation of the motion-activity extraction module 
15 24 of Figure 6, which appears as step S270 of Figure 7, is described with reference to 
Figure 8. In this process, for the input current frame and the preceding frame, 
differences are determined between the colour values of adjacent pixels in tune (step 
S271). Hie probability or relative frequency ofeach colour value difference in tune is 
determined (step S272). For each colour value difference in time a product of the 
20 probability of that difference and the natural logarithm of the probability of that 

difference, is determined (step S273). These products are simmied for all colour value 
differences in time (step S274), with the negative of that sum is output (step S275) as the 
motion-activity measure M. 

25 2fii^ Video Quality Measurement 



The motion-activity measure M is incorporated into the video qualily model by 
computing the quality score for each individual image in the video (i.e, image sequence) 
using the following video quality model: 
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The motion-activity measure M modulates the blurring effect since it has been 
observed that when more motion occurs in the video, human eyes tend to be less 
sensitive to higher blurring effects. 

The parameters of the video quality model can be estimated by fitting the model 
to subjective test data of video sequences, in a similar manner to the approach for the 
image quality model in the embodiment of Figure 1- 

Video quality measurement is achieved in the second embodiment by 
determining the quality score Qv of individual images in the unage sequence, and then 
combming the individual image quality scores Qy, to give a single video quality score Q 



where N is the total nimniber of frames over which Q is being computed (it is the last 
score of N at step S214 of Figure 7). 

The above first ^bodiment is used for measuring image quality of a single 



measuring the overall video quality of a video sequence. The system of the first 
embodiment may be used to measure video quality by averaging the image qualily 
measures over the number of frames of the video. In effect this is the same as the second 
embodiment, but wilhout the motion-activity feature extraction module 24 or the motion- 
activity measure M, 

Both the above-described embodiments use two new global no-reference image- 
quality features suitable for applications in non-reference objective image and video 
quality measurement systems: (1) image colour richness and (2) image sharpness- 
Further the second embodiment provides a new global no-reference video-quality feature 



as follows: 




image or of a frame in a video sequence, while the second embodiment is used for 
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suitable for applications in no-reference objective video quality measurement systems: 
(3) motion-activity. In addition, both above embodiments include an improved measure 
for measuring image blockiness, the image blockiness invisibility feature. 

5 The above-described embodiments provide new formulae to measure visual 

quality, one for images, using the two new no-reference image-quality features together 
with the improved measure of the image blockiness, the other for video, usmg the two 
new no-reference image-quality features and the new no-reference video-quality feature, 
together with the improved measure of the image blockiness, 

10 

These three new image/video features are unique in that they give values which 
are related to the perceived visual quality when distortions have been introduced into an 
original undistorted image (due to various processes such as image/video compressions 
and various forms of blurring etc). The computation of these image/video features 
1 5 requires the distorted image/video itself without any need for a reference undistorted 
image/video to be available (hence the term '*no-reference*'). 

The image colour richness feature measures the richness of an image's content 
and gives more colourM images hi^er values and dull images lower values. The image 
20 sharpness feature measures the sharpness of an image's content and assigns lower values 
to blurred images (due to smoothing or motion-blurring etc) and higher values to sharp 
images. Hie motion-activity feature measures the contribution of the motion in the video 
to the perceived image quality. The image blockiness invisibility feature provides an 
improved measure for measuring image blockiness. 

25 

The above embodiments are able to qualify images and video correctly, even 
those that may have been subjected to various forms of distortions, such as various types 
of image/video compressions (e.g. by JPEG compression based on DCTs or JPEG-2000 
compression based on wavelets, etc*) and also various form of blurring (e.g. by 
30 smoothing or motion-blurring). The results from the above-described embodiments of 
image/video quality measurement systems achieve a close correlation with respect to 
human visual subjective ratings, measured in terms of Pearson correlation or Spearman 
rank-order correlation. 



21 



Although in the above embodiments the various features as described are used in 
combination, individual ones or two or more of those features may be taken and used 
independently of the rest, for instance with other features instead. Likewise, additional 
5 features may be added to the above described systems* 

In the above description, components of the system are described as modules. A 
module, and in particular its functionality, can be implemented in either hardware or 
software or both. In the software sense, a module is a process, program, or portion 

10 thereof, that usually performs a particular fiinction or related ftmctions. In the hardware 
sense, a module is a ftinctional hardware unit designed for use with other components or 
modules. For example, a module may be implemented using discrete electronic 
components, or it can form a portion of an entire electronic circmt such as an Application 
Specific Integrated Circuit (ASIC). In a hardware and software sense, a module may be 

1 5 implemented as a processor, for instance a microprocessor, operating or operable 

according to the software in memory. Numerous other possibilities exist. Those skilled 
in the art will appreciate that the system can also be implemented as a combmation of 
hardware and software modules. 

20 The above described embodiments are directed toward measuring the quality of 

an image or video. The embodunents of the invention are able to do so using several 
variants in implementation. From the above description of a specific embodiment and 
alternatives, it will be apparent to those skilled in the art that modifications/changes can 
be made without departing from the scope and spirit of the invention. In addition, the 

25 general principles defined herein may be applied to other embodiments and applications 
without moving away from the scope and spirit of the invention. Consequentiy, the 
present invention is not intended to be limited to the embodiments shown, but is to be 
accorded the widest scope consistent with the principles and features disclosed herein. 
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CLAIMS 

Apparatus for determining a measure of image quality of an image, 
comprising: 

means for determining a blockiness invisibility measure of the image; 

means for determining a colour richness measure of the image; 

means for determining a sharpness measure of the image; and 

means for providing the measure of image quality of the image based on 
the blockiness invisibility measure, the colour richness measure and the sharpness 
measxjre of the image* 

Apparatus according to claim 1, wherein the means for determining the 
colour richness measure of the image is operable to provide the colour richness 
based on the sum of the products of the probabilities of colour values and the 
logarithms of those probabilities. 

Apparatus according to claim 1 or 2, wherein the means for determining 
the sharpness measure of the image is operable to provide the sharpness based on 
the sum of liie products of the probabilities of differences between neighbouring 
portions of the image and the logarithms of those probabilities- 
Apparatus according to claim 3» wherein the differences between 
neighbouring portions of the hnage are differences in colour values. 

Apparatus according to clafan 3 or 4, wherein the differences between 
neighbouring portions of the image are differeaaces in image data between 
neighbouring pixels. 

Apparatus for determining a blockiness invisibility measure of an image, 
comprising: 

means for averaging differences in colour values at block boundaries 
within the image; 
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means for averaging differences in colour values between adjacent pixels; 

and 

means for providing the blockiness invisibility measure based on 
averaged differences in colour values between adjacent pixels and averaged 
5 differences in colour values at block boundaries within the image. 

7. Apparatus for determining a colour richness measure of an image, 
comprising: 

means for determining the probabilities of individual colour values within 
10 the image; 

means for determining the products of the probabilities of individual 
colour values and the logarithms of the probabilities of individual colour values; 
and 

means for providing the colour richness measure based on the sum of the 
1 5 products of the probabilities of individual colour values and the logarithms of the 

probabilities of individual colour values* 

8. Apparatus for determining a sharpness measure of an image, comprising: 
means for determining differences in colour values between adjacent 

20 pixels within the image; 

means for determining the probabilities of individual colour value 
differences within the image; 

means for determining Ae products of the probabilities of individual 
colour value differences and the logarithms of the probabilities of individxial 
25 colour value diflFerences; and 

means for providing the sharpness measure based on the sum of the 
products of the probabilities of individual colour value differences and the 
logarithms of the probabilities of individual colour value differences. 
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Apparatus according to any one of claims 1 to 5, wherein the means for 
determining a blockiness invisibility measure of the image comprises apparatus 
according to claim 6. 
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Apparatus according to any one of claims 1 to 5 and 9, wherein the means 
for determining a colour richness measure of the image comprises apparatus 
according to claim 7. 

Apparatus according to any one of claims 1 to 5, 9 and 10, wherein the 
means for determining a sharpness measure of the image comprises apparatus 
according to claim 8. 

Apparatus for determining a measure of image quality of an image within 

a sequence of two or more images, comprising: 

apparatus according to any one of claims 1 to 5 and 9 to 1 1 ; and 

means for determining a motion activity measure of the image within the 

sequence of images. 

Apparatus for determining a motion activity measure of an image within a 
sequence of two or more images, comprising: 

means for determining differences in colour values between pixels within 
the image and corresponding pixels in a preceding image within the sequence of 
images; 

means for determining the probabilities of individual colour value 
differences between the image and the preceding image; 

means for determinmg the products of the probabilities of individual 
colour value differences and the logarithms of the probabilities of individual 
colour value differences; and 

means for providing the motion activity measure based on the sum of the 
products of the probabilities of individual colour value differences and the 
logarifhnis of Ihe probabilities of individual colour value differences. 

Apparatus according to claim 12, wherein the means for determining a 
motion activity measure of the image within the sequence of images comprises 
apparatus according to claim 13, 
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Apparatus according to claim 12 or 14, wherein the means for providing 
the measure of image quality of liie image is operable to provide the image 
quality measure further based on the motion activity measure of the image. 

Apparatus for determining a measure of video quality of a sequence of 

two or more images, comprising: 

apparatus according to any one of claims 1 to 5, 9 to 12, 14 and 15; and 
means for providing the measure of video quality based on an average of 

the image quality for a plurality of images within the sequence of two or more 

images. 

Apparatus according to any one of the preceding claims, operable to make 
the determination without reference to a reference image, 

A method of determining a measure of image quality of an image, 
comprising: 

determining a blockiness invisibility measure of the image; 

determining a colour richness measure of the image; 

determining a sharpness measure of the image; and 

provicKng the measure of image qviality of the image based on the 
blockiness invisibility measure, the colour richness measure and the sharpness 
measure of the image. 

A method according to claim 18, wherein determining the colour richness 
measvixe of the image comprises providing the colour richness based on the sum 
of the products of the probabilities of colour values and the logarithms of those 
probabilities. 

A mefliod according to clmm 18 or 19, wherein determining the sharpness 
measure of the image comprises providing the sharpness based on the sum of the 
products of the probabilities of differences between neighbouring portions of the 
image and the logarithms of those probabilities. 



26 

A method according to claim 20, wherein the diflferences between 
neighbouring portions of the image are dijfferences in colour values. 

A method according to claim 20 or 21, wherein the differences between 
neighbouring portions of the image are differences in image data between 
neighboxiring pixels. 

A method for determining a blockiness invisibility measure of an image, 
comprising: 

averaging differences in colour values at block boundaries within the 

image; 

averaging differences in colour values between adjacent pixels; and 
providing the blockiness invisibility measure based on averaged 

differences in colour values between adjacent pixels and averaged differences in 

colour values at block boundaries within the image. 

A method for determining a colour richness measure of an image, 
comprising: 

determining the probabilities of individual colour values within the image; 

determining the products of the probabilities of individual colour values 
and the logarithms of the probabilities of individual colour values; and 

providing the colour richness measure based on the sum of the products of 
the probabilities of individual colour values and the logarithms of the 
probabilities of individual colour values, 

A method for determining a sharpness measure of an image, comprising: 
determining differences in colour values between adjacent pixels wiHiin 
the image; 

determining the probabilities of individual colour value differences within 
the image; 

determining the products of the probabilities of individual colour value 
differences and the logarithms of the probabilities of individual colour value 
differences; and 
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providing the sharpness measure based on the sum of the products of the 
probabilities of individual colour value differences and the logarithms of the 
probabilities of individual colour value differences. . 

A method according to any one of claims 1 8 to 22, wherein detemiining a 
blockiness invisibility measure of the image comprises a method according to 
claim 23. 

A method according to any one of claims 18 to 22 and 26, wherein 
determining a colour richness measure of the image comprises a method 
according to claim 24. 

A method according to any one of claims 18 to 22, 26 and 27, wherein 
detenmning a sharpness measure of the image comprises a method according to 
claim 25, 

A method for determining a measure of image qualily of an image within 

a sequence of two or more images, comprising: 

a mefliod according to any one of claims 18 to 22 and 26 to 28; and 
determining a motion activity measure of the image within the sequence 

of images. 

A method for determining a motion activity measure of an image within a 
sequence of two or more images, comprising: 

determining differences in colour values between pixels within the image 
and corresponding pixels in a preceding image within the sequence of images; 

detemiining the probabilities of individual colour value differences 
between the unage and the preceding image; 

detemiining the products of the probabilities of individual colour value 
differences and the logarithms of the probabilities of individual colour value 
differences; and 
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providing the motion activity measure based on the sum of the products of 
tibie probabilities of individual colour value differences and the logarithms of the 
probabilities of individual colour value differences. 

5 31, A method according to claim 29, wherein determining a motion activity 

measure of the image within the sequence of images comprises a method 
according to claim 29. 

32. A method according to claim 29 or 3 1 , wherein providing the measure of 
10 image quality of the image comprises providing the image quality measure 

flirfher based on the motion activity measure of tiie image. 

33. A method for determining a measxure of video quality of a sequence of 
two or more images, comprising: 

15 a method according to any one of claims 1 8 to 22, 26 to 29, 3 1 and 32; 

and 

providing the measure of video quality based on an average of the image 
quality for a plurality of images within the sequence of two or more images. 

20 34. A method according to any one of the claims 18 to 33, wherein the 

determination is made without reference to a reference image. 

35. A method of determining a measure of video or unage quality 
substantially as hereinbefore described with reference to and as illustrated in the 

25 accompanying drawings. 

36. Apparatus according to any one of claims 1 to 17 operable in accordance 
with the method of any one of claims 18 to 35. 
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Apparatus for determining a measure of video or image quality 
constructed and arranged substantially as hereinbefore described with reference 
to and as illustrated in the accompanying drawings. 
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A computer program product having a computer usable medium having a 
computer readable program code means embodied therein for determining a 
measure of video or image quality, the computer program product comprising: 

computer readable program code means for operating according to the 
method of any one of claims 18 to 35. 

A computer program product having a computer usable medium having a 
computer readable program code means embodied therein for determining a 
measure of video or image quality, the computer program product comprising: 

computer readable program code means which, when downloaded onto a 
computer renders the computer into apparatus according to any one of claims 1 to 
17, 36 and 37. 



30 



IMAGE AND VIDEO QUALITY MEASUREMENT 

ABSTRACT 

5 An image quality measurement system (10) deteraiines various features of an 

image that relate to the quality of the image in terms of its appearance. The features 
include the image's biockiness invisibility (B), llie imagers colour richness (R) and the 
image's sharpness (S). These are all obtained without use of a reference image. The 
determined features are combined to provide an image quality measure (Q)* 
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